A systematic review of short-term and working memory abilities in children with intellectual disability; to what extent do the result support the delay or difference hypotheses?
Aim
The aim of the study is to investigate if short-term and working memory abilities exhibit a developmentally delayed or developmentally different pattern in children with ID. This will be done by an overall analysis and subgroup analyses based on type of memory (short-term or working memory), aetiology of ID (familial or organic). In addition, the overall analysis will also be repeated with level of IQ as a moderator.
Research questions
- Do short-term and working memory abilities exhibit a developmentally delayed or developmentally different pattern in children with ID (compared to mental age matched groups)? 1.1 Is this pattern moderated by origin of ID (familial and organic) or level of IQ?
- Do origin (familial and organic) of ID or level of IQ moderate whether STM and WM performance are delayed or different?
Hypotheses
The STM delay hypothesis will be supported if STM performance of participants with ID will not differ significantly from that of the mental age-matched group
The WM delay hypothesis will be supported if WM performance of participants with ID will not differ significantly from that of the mental age-matched group
The STM difference hypothesis will be supported if STM performance of participants with ID will differ significantly from that of the mental age-matched group. We expect that the difference may occur in any direction.
The WM difference hypothesis will be supported if WM performance of participants with ID will differ significantly from that of the mental age-matched group. We expect that the difference may occur in any direction.
Methods
Data extraction
We extracted data from stuff
Statistical analysis
We tested the difference in performance moderating for type of task and also for type of ID
Equivalence testing
Yes
Results
Descriptives
We conducted and reported this systematic review according to NIRO-SR guidelines for conducting and reporting systematic reviews of non-intervention research (REF). See table X for descriptives of the obtained studies.
| Name | data[, columns_of_interes… |
| Number of rows | 99 |
| Number of columns | 14 |
| _______________________ | |
| Column type frequency: | |
| numeric | 14 |
| ________________________ | |
| Group variables | None |
Variable type: numeric
| skim_variable | n_missing | complete_rate | mean | sd | p0 | p25 | p50 | p75 | p100 | hist |
|---|---|---|---|---|---|---|---|---|---|---|
| study_year | 0 | 1.00 | 2009.13 | 11.15 | 1962.00 | 2010.00 | 2012.00 | 2013.00 | 2020.00 | ▁▁▁▁▇ |
| n_id | 0 | 1.00 | 22.46 | 11.19 | 10.00 | 15.00 | 19.00 | 26.00 | 68.00 | ▇▆▁▁▁ |
| age_id | 3 | 0.97 | 16.31 | 6.24 | 4.34 | 13.59 | 14.98 | 17.73 | 49.88 | ▃▇▁▁▁ |
| mean_ma_id | 2 | 0.98 | 6.35 | 1.26 | 1.88 | 5.75 | 6.10 | 6.74 | 11.17 | ▁▂▇▁▁ |
| sd_ma_id | 29 | 0.71 | 2.06 | 4.82 | 0.37 | 0.67 | 0.93 | 1.56 | 24.61 | ▇▁▁▁▁ |
| iq_id | 10 | 0.90 | 48.78 | 8.50 | 19.92 | 44.17 | 53.00 | 53.30 | 65.75 | ▁▂▂▇▁ |
| n_magroup | 0 | 1.00 | 25.78 | 14.07 | 10.00 | 15.00 | 22.00 | 28.00 | 65.00 | ▇▅▁▁▁ |
| age_magroup | 9 | 0.91 | 6.57 | 1.32 | 1.93 | 5.75 | 6.27 | 7.55 | 11.17 | ▁▂▇▃▁ |
| mean_ma_magroup | 13 | 0.87 | 6.58 | 1.24 | 1.90 | 5.98 | 6.71 | 6.90 | 9.76 | ▁▂▇▇▂ |
| iq_magroup | 50 | 0.49 | 102.73 | 8.78 | 94.00 | 95.70 | 95.70 | 109.00 | 123.56 | ▇▁▅▁▁ |
| mean_id | 0 | 1.00 | 53.96 | 164.97 | 0.04 | 3.84 | 9.93 | 22.74 | 1096.20 | ▇▁▁▁▁ |
| mean_magroup | 0 | 1.00 | 52.19 | 165.84 | 0.03 | 3.78 | 9.67 | 22.81 | 1258.00 | ▇▁▁▁▁ |
| sd_id | 0 | 1.00 | 15.64 | 36.80 | 0.19 | 2.38 | 5.01 | 10.50 | 251.50 | ▇▁▁▁▁ |
| sd_magroup | 0 | 1.00 | 12.10 | 32.96 | 0.13 | 1.90 | 3.50 | 5.52 | 277.90 | ▇▁▁▁▁ |
For both groups, we extracted the iq, number of participants, mean and sd of the performance in the tasks, and the mean and sd of the chronological and mental age. There was a total of an 8.4% of missing data with the iq of the control group missing 51% of the times
Statistical analysis
We are performing a multilevel analysis
overall_3_level <- rma.mv(yi, vi, random = list(~ 1 | effectsize_id, ~1 | study_id),
tdist = TRUE, data = stats_test)
#two-level model without within-study variance (to estimate the between-study variance)
modelnovar2 <- rma.mv(yi, vi, random = list(~ 1 | effectsize_id, ~1 | study_id),
sigma2 = c(0,NA), tdist = TRUE, data = stats_test)
anova(overall_3_level,modelnovar2)##
## df AIC BIC AICc logLik LRT pval QE
## Full 3 287.2253 294.9802 287.4807 -140.6127 434.0747
## Reduced 2 414.6042 419.7741 414.7305 -205.3021 129.3788 <.0001 434.0747
#build a two-level model without between-study variance (to estimate the within-study variance);
modelnovar3 <- rma.mv(yi, vi, random = list(~ 1 | effectsize_id, ~ 1 | study_id),
sigma2 = c(NA,0), tdist = TRUE, data = stats_test)
anova(overall_3_level,modelnovar3)##
## df AIC BIC AICc logLik LRT pval QE
## Full 3 287.2253 294.9802 287.4807 -140.6127 434.0747
## Reduced 2 285.2253 290.3953 285.3517 -140.6127 0.0000 1.0000 434.0747
Moderating type of ID
res_DS <- rma(yi, vi, data=stats_test, subset=group_id=="DS")
res_NSID <- rma(yi, vi, data=stats_test, subset=group_id=="NSID")
res_FXS <- rma(yi, vi, data=stats_test, subset=group_id=="FXS")
res_WS <- rma(yi, vi, data=stats_test, subset=group_id=="WS")
dat.comp <- data.frame(alloc = c("DS", "NSID", "FXS", "WS"),
estimate = c(coef(res_DS), coef(res_NSID), coef(res_FXS), coef(res_WS)),
stderror = c(res_DS$se, res_NSID$se, res_FXS$se, res_WS$se),
tau2 = c(res_DS$tau2, res_NSID$tau2, res_FXS$tau2, res_WS$tau2))
dfround(dat.comp, 3)## alloc estimate stderror tau2
## 1 DS -0.175 0.119 0.301
## 2 NSID -0.106 0.120 0.243
## 3 FXS -0.287 0.526 1.550
## 4 WS 0.044 0.210 1.120
##
## Fixed-Effects with Moderators Model (k = 4)
##
## I^2 (residual heterogeneity / unaccounted variability): 0.00%
## H^2 (unaccounted variability / sampling variability): 1.00
## R^2 (amount of heterogeneity accounted for): NA%
##
## Test for Residual Heterogeneity:
## QE(df = 0) = 0.000, p-val = 1.000
##
## Test of Moderators (coefficients 2:4):
## QM(df = 3) = 0.939, p-val = 0.816
##
## Model Results:
##
## estimate se zval pval ci.lb ci.ub
## intrcpt -0.175 0.119 -1.474 0.140 -0.408 0.058
## allocFXS -0.112 0.539 -0.208 0.835 -1.169 0.945
## allocNSID 0.069 0.169 0.409 0.682 -0.262 0.400
## allocWS 0.219 0.241 0.908 0.364 -0.254 0.692
##
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
id_tests <- t(data.frame(DS = c(res_DS$b, res_DS$se,res_DS$ci.lb, res_DS$ci.ub),
NSID = c(res_NSID$b, res_NSID$se,res_NSID$ci.lb, res_NSID$ci.ub),
FXS = c(res_FXS$b, res_FXS$se,res_FXS$ci.lb, res_FXS$ci.ub),
WS = c(res_WS$b, res_WS$se,res_WS$ci.lb, res_WS$ci.ub),
row.names = c('b', 'se', 'ci.lb', 'ci.ub')))
for (id in c('DS', 'NSID', 'FXS', 'WS')){
df <- stats_test[stats_test$group_id == id, c('cite', 'yi', 'sei', 'n_magroup', 'n_id')]
# Compute confidence intervals and format labels
df$cite <- paste0(df$cite, '.', ave(df$cite, df$cite, FUN = seq_along))
df <- df %>%
mutate(
lower = yi - 1.96 * sei,
upper = yi + 1.96 * sei,
ci_text = paste0(round(yi,2), " [", round(lower, 2), ", ", round(upper, 2), "]")
)
FE_model <- paste0('F.E Model for ', id, ' group')
df <- df %>%
arrange(yi)
df <- rbind(df, c(FE_model, id_tests[id,'b'], id_tests[id,'se'], '','',
id_tests[id,'ci.lb'], id_tests[id,'ci.ub'],
paste0(round(id_tests[id,'b'],2),
" [", round(id_tests[id,'ci.lb'], 2), ", ",
round(id_tests[id,'ci.ub'], 2), "]")))
df[, c('yi','sei', 'lower', 'upper')] <- lapply(df[, c('yi','sei', 'lower', 'upper')], as.numeric)
df$desired_order <- seq(dim(df)[1],1)
p_right <- ggplot(df, aes(y = reorder(cite, desired_order))) +
geom_point(aes(x=yi), size = 3, shape=15, color = "darkred") +
geom_errorbarh(aes(xmin = lower, xmax = upper), height = 0.2, color = "grey") +
labs(
x = "Effect Size (Cohen's d)",
y = "Study",
caption = "Error bars represent 95% confidence intervals"
) + theme(axis.line.y = element_blank(),
axis.ticks.y= element_blank(),
axis.text.y= element_blank(),
axis.title.y= element_blank(),
panel.background = element_rect(fill = "white"
)) + coord_cartesian(xlim = c(min(df$lower), max(df$upper)), ylim = c(0,dim(df)[1] + 1))
p_left <- ggplot(df, aes(y = reorder(cite, desired_order))) + labs(x = '') +
geom_text(aes(x = 0, label = cite), hjust = 0, fontface = "bold") +
annotate(geom="text", x=0.3, y=dim(df)[1] + 1, label="Study", fontface = "bold") +
geom_text(aes(x = 0.8, label = n_id), hjust = 0) +
annotate(geom="text", x=0.8, y=dim(df)[1] + 1, label="N (ID)", fontface = "bold") +
geom_text(aes(x = 1, label = n_magroup), hjust = 0) +
annotate(geom="text", x=1, y=dim(df)[1] + 1, label="N (Control)", fontface = "bold") +
geom_text(
aes(x = 1.3, label = ci_text),
hjust = 0
) +
annotate(geom="text", x=1.4, y=dim(df)[1] + 1, label="E.F [95% CI]", fontface = "bold") +
theme_void() +
coord_cartesian(xlim = c(0, 2), ylim = c(0,dim(df)[1] + 1))
layout <- c(
area(t = 0, l = 0, b = dim(df)[1], r = 5),
area(t = 0, l = 6, b = dim(df)[1], r = 8)
)
print(p_left + p_right + plot_layout(design = layout))
Sys.sleep(2)
}Moderating type of executive function
res_inhibition <- rma(yi, vi, data=stats_test, subset=ef_type=="inhibition")
res_updating <- rma(yi, vi, data=stats_test, subset=ef_type=="updating")
res_shifting <- rma(yi, vi, data=stats_test, subset=ef_type=="shifting")
res_fluency <- rma(yi, vi, data=stats_test, subset=ef_type=="fluency")
res_attention <- rma(yi, vi, data=stats_test, subset=ef_type=="attention")
res_other <- rma(yi, vi, data=stats_test, subset=ef_type=="other")
dat.comp <- data.frame(alloc = c("Inhibition", "Updating", "Shifting", "Fluency", 'Attention', 'Other'),
estimate = c(coef(res_inhibition), coef(res_updating), coef(res_shifting), coef(res_fluency), coef(res_attention), coef(res_other)),
stderror = c(res_inhibition$se, res_updating$se, res_shifting$se, res_fluency$se, res_attention$se, res_other$se),
tau2 = c(res_inhibition$tau2, res_updating$tau2, res_shifting$tau2, res_fluency$tau2, res_attention$tau2, res_other$tau2))
dfround(dat.comp, 3)## alloc estimate stderror tau2
## 1 Inhibition -0.105 0.180 0.900
## 2 Updating -0.549 0.275 0.000
## 3 Shifting 0.173 0.154 0.406
## 4 Fluency -0.108 0.207 0.379
## 5 Attention -0.054 0.203 0.437
## 6 Other -0.624 0.261 0.456
##
## Fixed-Effects with Moderators Model (k = 6)
##
## I^2 (residual heterogeneity / unaccounted variability): 0.00%
## H^2 (unaccounted variability / sampling variability): 1.00
## R^2 (amount of heterogeneity accounted for): NA%
##
## Test for Residual Heterogeneity:
## QE(df = 0) = 0.000, p-val = 1.000
##
## Test of Moderators (coefficients 2:6):
## QM(df = 5) = 9.882, p-val = 0.079
##
## Model Results:
##
## estimate se zval pval ci.lb ci.ub
## intrcpt -0.054 0.203 -0.268 0.788 -0.452 0.343
## allocFluency -0.053 0.290 -0.184 0.854 -0.621 0.515
## allocInhibition -0.051 0.272 -0.187 0.852 -0.583 0.481
## allocOther -0.570 0.331 -1.723 0.085 -1.218 0.079 .
## allocShifting 0.228 0.255 0.893 0.372 -0.272 0.727
## allocUpdating -0.495 0.342 -1.448 0.148 -1.165 0.175
##
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
id_tests <- t(data.frame(inhibition = c(res_inhibition$b, res_inhibition$se,res_inhibition$ci.lb, res_inhibition$ci.ub),
updating = c(res_updating$b, res_updating$se,res_updating$ci.lb, res_updating$ci.ub),
shifting = c(res_shifting$b, res_shifting$se,res_shifting$ci.lb, res_shifting$ci.ub),
fluency = c(res_fluency$b, res_fluency$se,res_fluency$ci.lb, res_fluency$ci.ub),
attention = c(res_attention$b, res_attention$se,res_attention$ci.lb, res_attention$ci.ub),
other = c(res_other$b, res_other$se,res_other$ci.lb, res_other$ci.ub),
row.names = c('b', 'se', 'ci.lb', 'ci.ub')))
for (id in c("inhibition", "updating", "shifting", "fluency", 'attention', 'other')){
df <- stats_test[stats_test$ef_type == id, c('cite', 'yi', 'sei', 'n_magroup', 'n_id')]
# Compute confidence intervals and format labels
df$cite <- paste0(df$cite, '.', ave(df$cite, df$cite, FUN = seq_along))
df <- df %>%
mutate(
lower = yi - 1.96 * sei,
upper = yi + 1.96 * sei,
ci_text = paste0(round(yi,2), " [", round(lower, 2), ", ", round(upper, 2), "]")
)
FE_model <- paste0('F.E Model for ', id, ' group')
df <- df %>%
arrange(yi)
df <- rbind(df, c(FE_model, id_tests[id,'b'], id_tests[id,'se'], '','',
id_tests[id,'ci.lb'], id_tests[id,'ci.ub'],
paste0(round(id_tests[id,'b'],2),
" [", round(id_tests[id,'ci.lb'], 2), ", ",
round(id_tests[id,'ci.ub'], 2), "]")))
df[, c('yi','sei', 'lower', 'upper')] <- lapply(df[, c('yi','sei', 'lower', 'upper')], as.numeric)
df$desired_order <- seq(dim(df)[1],1)
p_right <- ggplot(df, aes(y = reorder(cite, desired_order))) +
geom_point(aes(x=yi), size = 3, shape=15, color = "darkred") +
geom_errorbarh(aes(xmin = lower, xmax = upper), height = 0.2, color = "grey") +
labs(
x = "Effect Size (Cohen's d)",
y = "Study",
caption = "Error bars represent 95% confidence intervals"
) + theme(axis.line.y = element_blank(),
axis.ticks.y= element_blank(),
axis.text.y= element_blank(),
axis.title.y= element_blank(),
panel.background = element_rect(fill = "white"
)) + coord_cartesian(xlim = c(min(df$lower), max(df$upper)), ylim = c(0,dim(df)[1] + 1))
p_left <- ggplot(df, aes(y = reorder(cite, desired_order))) + labs(x = '') +
geom_text(aes(x = 0, label = cite), hjust = 0, fontface = "bold") +
annotate(geom="text", x=0.3, y=dim(df)[1] + 1, label="Study", fontface = "bold") +
geom_text(aes(x = 0.8, label = n_id), hjust = 0) +
annotate(geom="text", x=0.8, y=dim(df)[1] + 1, label="N (ID)", fontface = "bold") +
geom_text(aes(x = 1, label = n_magroup), hjust = 0) +
annotate(geom="text", x=1, y=dim(df)[1] + 1, label="N (Control)", fontface = "bold") +
geom_text(
aes(x = 1.3, label = ci_text),
hjust = 0
) +
annotate(geom="text", x=1.4, y=dim(df)[1] + 1, label="E.F [95% CI]", fontface = "bold") +
theme_void() +
coord_cartesian(xlim = c(0, 2), ylim = c(0,dim(df)[1] + 1))
layout <- c(
area(t = 0, l = 0, b = dim(df)[1], r = 5),
area(t = 0, l = 6, b = dim(df)[1], r = 8)
)
print(p_left + p_right + plot_layout(design = layout))
Sys.sleep(2)
}
### Equivalence testing The studies looked at different types of
executive functions (CHANGE THIS LATER TO SUIT OUR OWN ANALYSIS). In the
next table you can see the extracted descriptives separated by executive
function
##
## Descriptive statistics by group
## group: attention
## vars n mean sd median trimmed mad min max
## study_year 1 16 2011.94 2.24 2012.00 2011.57 1.48 2010.00 2019.00
## n_id 2 16 22.06 9.44 15.00 21.29 0.00 15.00 40.00
## age_id 3 16 16.51 5.08 17.60 16.19 2.92 9.59 27.81
## mean_ma_id 4 16 5.74 0.82 5.92 5.77 0.86 4.47 6.70
## sd_ma_id 5 10 0.84 0.30 0.90 0.83 0.15 0.37 1.38
## iq_id 6 12 46.37 12.02 53.00 48.32 0.44 19.92 53.30
## n_magroup 7 16 24.38 10.97 16.00 23.93 1.48 15.00 40.00
## age_magroup 8 16 6.20 1.45 6.58 6.24 1.52 4.15 7.60
## mean_ma_magroup 9 13 6.06 1.15 6.90 6.13 0.00 4.45 6.90
## iq_magroup 10 8 94.85 0.91 94.85 94.85 1.26 94.00 95.70
## mean_id 11 16 26.95 34.79 10.06 22.99 6.62 2.13 107.30
## mean_magroup 12 16 26.12 33.38 8.50 22.80 3.72 1.96 96.80
## sd_id 13 16 6.90 7.16 3.72 6.05 4.43 0.42 25.20
## sd_magroup 14 16 5.03 6.51 1.64 4.29 1.80 0.35 20.10
## range skew kurtosis se
## study_year 9.00 1.79 3.42 0.56
## n_id 25.00 0.78 -1.04 2.36
## age_id 18.22 0.25 -0.62 1.27
## mean_ma_id 2.23 -0.38 -1.33 0.20
## sd_ma_id 1.01 -0.16 -0.83 0.10
## iq_id 33.38 -1.25 -0.16 3.47
## n_magroup 25.00 0.45 -1.70 2.74
## age_magroup 3.45 -0.33 -1.71 0.36
## mean_ma_magroup 2.45 -0.56 -1.72 0.32
## iq_magroup 1.70 0.00 -2.23 0.32
## mean_id 105.17 1.40 0.43 8.70
## mean_magroup 94.84 1.25 -0.14 8.34
## sd_id 24.78 1.10 0.18 1.79
## sd_magroup 19.75 1.35 0.15 1.63
## ------------------------------------------------------------
## group: fluency
## vars n mean sd median trimmed mad min max
## study_year 1 12 2011.17 4.57 2012.00 2011.10 1.48 2003.00 2020.00
## n_id 2 12 26.50 13.71 24.00 23.50 2.97 15.00 68.00
## age_id 3 12 14.31 1.41 13.64 14.05 0.70 13.17 18.10
## mean_ma_id 4 12 6.39 1.19 6.38 6.38 1.24 4.50 8.39
## sd_ma_id 5 10 1.15 0.36 1.10 1.16 0.54 0.67 1.58
## iq_id 6 12 45.00 9.26 45.26 45.30 10.91 30.65 56.33
## n_magroup 7 12 26.50 12.03 26.00 24.10 4.45 15.00 62.00
## age_magroup 8 12 6.31 0.96 6.14 6.30 1.17 4.92 7.84
## mean_ma_magroup 9 10 6.77 1.42 6.53 6.83 2.19 4.50 8.55
## iq_magroup 10 3 109.00 0.00 109.00 109.00 0.00 109.00 109.00
## mean_id 11 12 17.78 9.16 16.60 16.79 10.85 7.13 38.30
## mean_magroup 12 12 18.01 8.74 14.99 17.11 9.01 8.00 37.00
## sd_id 13 12 7.50 2.76 8.59 7.52 3.42 2.77 12.10
## sd_magroup 14 12 5.74 1.65 5.31 5.68 1.55 3.26 8.73
## range skew kurtosis se
## study_year 17.00 -0.33 -0.13 1.32
## n_id 53.00 2.20 4.07 3.96
## age_id 4.93 1.47 1.45 0.41
## mean_ma_id 3.89 -0.14 -1.13 0.34
## sd_ma_id 0.92 0.00 -1.73 0.12
## iq_id 25.68 -0.22 -1.41 2.67
## n_magroup 47.00 1.98 3.47 3.47
## age_magroup 2.92 0.09 -1.37 0.28
## mean_ma_magroup 4.05 -0.41 -1.30 0.45
## iq_magroup 0.00 NaN NaN 0.00
## mean_id 31.17 0.69 -0.49 2.65
## mean_magroup 29.00 0.68 -0.73 2.52
## sd_id 9.33 -0.12 -1.32 0.80
## sd_magroup 5.47 0.28 -1.35 0.48
## ------------------------------------------------------------
## group: inhibition
## vars n mean sd median trimmed mad min max
## study_year 1 37 2011.95 3.94 2013.00 2012.29 1.48 2001.00 2018.00
## n_id 2 37 20.70 8.11 19.00 20.13 7.41 11.00 42.00
## age_id 3 37 16.64 7.91 14.68 15.30 1.70 4.34 49.88
## mean_ma_id 4 35 6.51 1.71 6.10 6.44 0.82 1.88 11.17
## sd_ma_id 5 24 4.10 7.93 1.10 2.40 0.74 0.54 24.61
## iq_id 6 34 48.88 8.41 53.00 49.48 9.04 30.65 59.10
## n_magroup 7 37 27.30 16.24 21.00 24.90 8.90 13.00 65.00
## age_magroup 8 34 6.63 1.63 6.14 6.60 1.76 1.93 11.17
## mean_ma_magroup 9 32 6.53 1.48 6.53 6.60 1.25 1.90 9.76
## iq_magroup 10 18 103.57 8.09 107.20 102.92 4.37 94.00 123.56
## mean_id 11 37 64.21 189.01 7.08 13.01 9.27 0.45 948.90
## mean_magroup 12 37 57.89 161.58 7.00 12.47 9.00 0.25 706.70
## sd_id 13 37 16.03 36.81 4.79 7.46 4.54 0.19 205.40
## sd_magroup 14 37 9.30 20.68 3.27 3.74 2.15 0.13 95.40
## range skew kurtosis se
## study_year 17.00 -0.83 0.82 0.65
## n_id 31.00 0.67 -0.55 1.33
## age_id 45.54 2.82 8.75 1.30
## mean_ma_id 9.30 0.51 1.43 0.29
## sd_ma_id 24.07 2.11 2.60 1.62
## iq_id 28.45 -0.57 -0.89 1.44
## n_magroup 52.00 1.36 0.48 2.67
## age_magroup 9.24 0.07 1.27 0.28
## mean_ma_magroup 7.86 -0.54 1.42 0.26
## iq_magroup 29.56 0.45 -0.34 1.91
## mean_id 948.45 3.46 11.67 31.07
## mean_magroup 706.45 3.04 7.88 26.56
## sd_id 205.21 3.94 16.47 6.05
## sd_magroup 95.27 3.08 8.43 3.40
## ------------------------------------------------------------
## group: other
## vars n mean sd median trimmed mad min max
## study_year 1 9 2013.00 4.03 2012.00 2013.00 2.97 2008.00 2020.00
## n_id 2 9 25.00 16.72 18.00 25.00 4.45 15.00 65.00
## age_id 3 9 17.58 5.72 17.60 17.58 2.24 10.10 30.30
## mean_ma_id 4 9 6.89 0.91 6.70 6.89 1.07 5.54 8.43
## sd_ma_id 5 6 1.15 0.44 1.17 1.15 0.51 0.52 1.58
## iq_id 6 8 53.52 3.22 53.30 53.52 2.45 46.57 56.33
## n_magroup 7 9 26.00 17.02 18.00 26.00 4.45 15.00 62.00
## age_magroup 8 8 7.51 0.18 7.50 7.51 0.15 7.33 7.84
## mean_ma_magroup 9 9 7.23 0.95 6.90 7.23 1.04 5.30 8.55
## iq_magroup 10 6 99.57 7.35 95.70 99.57 2.52 94.00 109.00
## mean_id 11 9 18.46 26.54 7.80 18.46 11.51 0.04 84.90
## mean_magroup 12 9 24.43 29.30 12.50 24.43 15.42 0.03 87.00
## sd_id 13 9 6.06 5.81 3.80 6.06 3.11 0.83 15.98
## sd_magroup 14 9 9.05 15.32 2.20 9.05 2.10 0.78 47.75
## range skew kurtosis se
## study_year 12.00 0.64 -1.11 1.34
## n_id 50.00 1.47 0.74 5.57
## age_id 20.20 0.87 0.12 1.91
## mean_ma_id 2.89 0.13 -1.34 0.30
## sd_ma_id 1.06 -0.20 -1.94 0.18
## iq_id 9.76 -1.01 -0.07 1.14
## n_magroup 47.00 1.17 -0.40 5.67
## age_magroup 0.51 0.49 -1.24 0.06
## mean_ma_magroup 3.25 -0.52 -0.60 0.32
## iq_magroup 15.00 0.51 -1.96 3.00
## mean_id 84.86 1.63 1.43 8.85
## mean_magroup 86.97 1.08 -0.37 9.77
## sd_id 15.15 0.84 -1.08 1.94
## sd_magroup 46.97 1.74 1.57 5.11
## ------------------------------------------------------------
## group: shifting
## vars n mean sd median trimmed mad min max
## study_year 1 24 2000.42 19.51 2010.00 2002.70 4.45 1962.00 2019.00
## n_id 2 24 22.25 12.95 15.50 20.35 7.41 10.00 67.00
## age_id 3 21 15.12 3.11 15.17 15.18 3.60 10.10 20.02
## mean_ma_id 4 24 6.31 0.70 6.10 6.24 0.52 5.27 8.33
## sd_ma_id 5 19 0.91 0.32 0.90 0.89 0.35 0.54 1.58
## iq_id 6 23 50.20 6.58 53.00 50.35 5.01 37.90 65.75
## n_magroup 7 24 23.83 13.12 18.00 22.10 5.93 10.00 62.00
## age_magroup 8 19 6.58 0.95 6.14 6.59 1.54 5.10 7.84
## mean_ma_magroup 9 21 6.63 0.82 6.53 6.58 0.55 5.30 8.55
## iq_magroup 10 14 106.18 10.60 109.00 106.19 13.79 94.00 118.30
## mean_id 11 24 89.61 235.97 8.98 29.25 8.93 0.81 1096.20
## mean_magroup 12 24 90.14 268.75 7.00 20.05 7.87 1.10 1258.00
## sd_id 13 24 29.02 57.31 3.19 15.11 3.14 0.85 251.50
## sd_magroup 14 24 25.77 59.82 3.35 11.11 2.18 1.30 277.90
## range skew kurtosis se
## study_year 57.00 -1.05 -0.76 3.98
## n_id 57.00 1.89 3.50 2.64
## age_id 9.92 0.00 -1.38 0.68
## mean_ma_id 3.06 1.06 0.71 0.14
## sd_ma_id 1.04 0.78 -0.42 0.07
## iq_id 27.85 -0.24 0.10 1.37
## n_magroup 52.00 1.31 0.92 2.68
## age_magroup 2.74 -0.13 -1.69 0.22
## mean_ma_magroup 3.25 0.61 -0.11 0.18
## iq_magroup 24.30 -0.09 -1.95 2.83
## mean_id 1095.39 3.37 11.08 48.17
## mean_magroup 1256.90 3.55 12.09 54.86
## sd_id 250.65 2.70 7.10 11.70
## sd_magroup 276.60 3.23 10.38 12.21
## ------------------------------------------------------------
## group: updating
## vars n mean sd median trimmed mad min max range
## study_year 1 1 2010.00 NA 2010.00 2010.00 0 2010.00 2010.00 0
## n_id 2 1 28.00 NA 28.00 28.00 0 28.00 28.00 0
## age_id 3 1 38.30 NA 38.30 38.30 0 38.30 38.30 0
## mean_ma_id 4 1 6.17 NA 6.17 6.17 0 6.17 6.17 0
## sd_ma_id 5 1 1.50 NA 1.50 1.50 0 1.50 1.50 0
## iq_id 6 0 NaN NA NA NaN NA Inf -Inf -Inf
## n_magroup 7 1 28.00 NA 28.00 28.00 0 28.00 28.00 0
## age_magroup 8 1 6.40 NA 6.40 6.40 0 6.40 6.40 0
## mean_ma_magroup 9 1 6.50 NA 6.50 6.50 0 6.50 6.50 0
## iq_magroup 10 0 NaN NA NA NaN NA Inf -Inf -Inf
## mean_id 11 1 5.04 NA 5.04 5.04 0 5.04 5.04 0
## mean_magroup 12 1 7.29 NA 7.29 7.29 0 7.29 7.29 0
## sd_id 13 1 3.76 NA 3.76 3.76 0 3.76 3.76 0
## sd_magroup 14 1 3.98 NA 3.98 3.98 0 3.98 3.98 0
## skew kurtosis se
## study_year NA NA NA
## n_id NA NA NA
## age_id NA NA NA
## mean_ma_id NA NA NA
## sd_ma_id NA NA NA
## iq_id NA NA NA
## n_magroup NA NA NA
## age_magroup NA NA NA
## mean_ma_magroup NA NA NA
## iq_magroup NA NA NA
## mean_id NA NA NA
## mean_magroup NA NA NA
## sd_id NA NA NA
## sd_magroup NA NA NA
Equivalence testing for the different types of tasks.
inhibition <- stats_test[stats_test$ef_type == 'inhibition',]
inhibition_model <- rma(yi, vi, data = inhibition)
inhibition_results <- ma_pipe_sei(
inhibition,
true_effect = inhibition_model$b,
rep_lower = inhibition_model$ci.lb,
rep_upper = inhibition_model$ci.ub,
analysis_title = "inhibition tasks",
plot = TRUE
)##
##
## NHST: don't reject null significance hypothesis that the effect is equal to 0
## TOST: don't reject null equivalence hypothesis
## Warning: Using `size` aesthetic for lines was deprecated in ggplot2 3.4.0.
## ℹ Please use `linewidth` instead.
## This warning is displayed once every 8 hours.
## Call `lifecycle::last_lifecycle_warnings()` to see where this warning was
## generated.
inhibition_power_median_dat <- inhibition_results$power_median_dat
inhibition_power_median_dat$ES <- mean(inhibition$yi)
inhibition_et_dat <- inhibition_results$et
updating <- stats_test[stats_test$ef_type == 'updating',]
updating_model <- rma(yi, vi, data = updating)
updating_results <- ma_pipe_sei(
updating,
true_effect = updating_model$b,
rep_lower = updating_model$ci.lb,
rep_upper = updating_model$ci.ub,
analysis_title = "updating tasks",
plot = TRUE
)##
##
## NHST: reject null significance hypothesis that the effect is equal to 0
## TOST: don't reject null equivalence hypothesis
updating_power_median_dat <- updating_results$power_median_dat
updating_power_median_dat$ES <- mean(updating$yi)
updating_et_dat <- updating_results$et
shifting <- stats_test[stats_test$ef_type == 'shifting',]
shifting_model <- rma(yi, vi, data = shifting)
shifting_results <- ma_pipe_sei(
shifting,
true_effect = shifting_model$b,
rep_lower = shifting_model$ci.lb,
rep_upper = shifting_model$ci.ub,
analysis_title = "shifting tasks",
plot = TRUE
)##
##
## NHST: don't reject null significance hypothesis that the effect is equal to 0
## TOST: don't reject null equivalence hypothesis
shifting_power_median_dat <- shifting_results$power_median_dat
shifting_power_median_dat$ES <- mean(shifting$yi)
shifting_et_dat <- shifting_results$et
fluency <- stats_test[stats_test$ef_type == 'fluency',]
fluency_model <- rma(yi, vi, data = fluency)
fluency_results <- ma_pipe_sei(
fluency,
true_effect = fluency_model$b,
rep_lower = fluency_model$ci.lb,
rep_upper = fluency_model$ci.ub,
analysis_title = "fluency tasks",
plot = TRUE
)##
##
## NHST: don't reject null significance hypothesis that the effect is equal to 0
## TOST: don't reject null equivalence hypothesis
fluency_power_median_dat <- fluency_results$power_median_dat
fluency_power_median_dat$ES <- mean(fluency$yi)
fluency_et_dat <- fluency_results$et
attention <- stats_test[stats_test$ef_type == 'attention',]
attention_model <- rma(yi, vi, data = attention)
attention_results <- ma_pipe_sei(
attention,
true_effect = attention_model$b,
rep_lower = attention_model$ci.lb,
rep_upper = attention_model$ci.ub,
analysis_title = "attention tasks",
plot = TRUE
)##
##
## NHST: don't reject null significance hypothesis that the effect is equal to 0
## TOST: don't reject null equivalence hypothesis
attention_power_median_dat <- attention_results$power_median_dat
attention_power_median_dat$ES <- mean(attention$yi)
attention_et_dat <- attention_results$et
other <- stats_test[stats_test$ef_type == 'other',]
other_model <- rma(yi, vi, data = other)
other_results <- ma_pipe_sei(
other,
true_effect = other_model$b,
rep_lower = other_model$ci.lb,
rep_upper = other_model$ci.ub,
analysis_title = "other tasks",
plot = TRUE
)##
##
## NHST: reject null significance hypothesis that the effect is equal to 0
## TOST: don't reject null equivalence hypothesis
other_power_median_dat <- other_results$power_median_dat
other_power_median_dat$ES <- mean(other$yi)
other_et_dat <- other_results$et
com1 <- rbind(inhibition_et_dat,
updating_et_dat,
shifting_et_dat,
fluency_et_dat,
attention_et_dat,
other_et_dat)
com1 <- combine_et(com1)
com1 <- com1 + ggtitle("Summary effect sizes and equivalence bounds") +
theme(plot.title = element_text(hjust = 0.5))
# Create the meta-analysis power tile supplement
power_med <- rbind(inhibition_power_median_dat,
updating_power_median_dat,
shifting_power_median_dat,
fluency_power_median_dat,
attention_power_median_dat,
other_power_median_dat)
power_med_long <- gather(power_med, effect, power, observed:e66, factor_key=TRUE)
tile <- ggplot(data = power_med_long, aes(x = effect, y = reorder(analysis, -ES),)) +
geom_tile(aes(fill = power)) +
coord_equal(ratio = 0.8) +
scale_fill_gradient(name = "Power", low = "#FDF0FF", high = "#E200FD") + theme_tufte(base_family="Helvetica")
tile <- tile + theme(axis.text.y = element_blank(),
axis.ticks.y = element_blank(),
axis.title.y = element_blank())
tile <- tile + ggtitle("Median statistical power") +
xlab("Effect size") +
theme(plot.title = element_text(hjust = 0.5))
tile <- tile + scale_x_discrete(labels=c("observed" = "Observed \n effect", "e33" = "0.33",
"e66" = "0.66"))
tile